Use of Colour and Shape Constraints in Vision-based Valve Operation by Robot

This paper proposes a new strategy for a humanoid robot to approach and operate a valve based on colour and shape constraints. It consists of four stages, namely rough base approaching, fine base approaching, rough hand approaching and fine hand approaching and grasping. The robot estimates the object’s position using its stereo-vision at the first stage. A new visual positioning method is used to determine the valve’s position and pose in the robot’s frame in the second stage. When its hands are near the valve, a visual servoing method is employed to catch the handle of the valve via cameras in end-effectors. The advantages of both eye-in-head and eye-to-hand systems are exploited. Experimental results are presented to verify the effectiveness of the proposed method.